\section{Evaluation}

We have implemented \sys by modifying HBase~\cite{hbase} and
HDFS~\cite{Shvachko10HDFS} to add pipelined commit,
active storage, and end-to-end checks. Our current implementation lags behind our design
in two ways.  First, our prototype supports unanimous consent between
HBase and HDFS but not between HBase and ZooKeeper. Second, while our
design calls for a BFT-replicated Master, NameNode, and ZooKeeper, our
prototype does not yet incorporate these features.  We intend to use
UpRight~\cite{clement09upright} to replicate NameNode, ZooKeeper, and
Master. Implementing these features would require a replicated state
machine (RSM) to communicate with another RSM, which looks simple but
actually is not, and we are currently working on this problem in another
project~\cite{Kapritsos14Adam}.

Our evaluation tries to answer two basic questions. First, does \sys provide
the expected guarantees despite a wide range of failures? Second, given its stronger guarantees, is
\sys' performance competitive with HBase?
Figure~\ref{graph:summary} summarizes the main results.

In the originally published version~\cite{yang13salus}, \sys
 did not incorporate witness nodes in the active
storage protocol and a combination of concurrent errors could cause it to
lose a suffix of data~\cite{yang13salus}. The active storage protocol presented
in this dissertation eliminates this problem and guarantees that no data
will be lost. However, since we don't have the
resources to rerun all our original experiments, most of the results presented
in this section are still obtained with the old protocol. We
add an experiment at the end of the section to measure the additional overhead of the new protocol.

\begin{figure}[t]
\begin{footnotesize}
\begin{center}
\begin{tabular}{ m{10cm} m{1cm} }
  \hline
  \foosys ensures \emph{freshness, \oc, and liveness} when there are no
  more than 2 failures within any \rrs and the corresponding \Dn{s}. & \S\ref{section:robustness} \\ \hline

  \foosys achieves comparable or better single-client throughput compared to HBase with slightly increased latency. & \S\ref{section:single-client} \\ \hline
  \foosys' active replication  can reduce network usage by 55\% and increase aggregate throughput by 74\% for sequential write workload compared to HBase.
  \foosys can achieve similar aggregate read throughput compared to HBase. & \S\ref{section:aggregate}  \\ \hline
  \foosys' overhead over HBase does not grow with the scale of the system. & \S\ref{sec:eval-scalability}  \\ \hline
\end{tabular}
\caption{\label{graph:summary} Summary of main results.}
\end{center}
\end{footnotesize}
\end{figure}


\input{salus_eval_robustness}
\input{salus_eval_performance}


%\input{lazy-recovery-eval}
%\subsection{Replication policy}
